Add chain syncing via compact block filters (BIP 157)#828
Draft
febyeji wants to merge 10 commits intolightningdevkit:mainfrom
Draft
Add chain syncing via compact block filters (BIP 157)#828febyeji wants to merge 10 commits intolightningdevkit:mainfrom
febyeji wants to merge 10 commits intolightningdevkit:mainfrom
Conversation
|
👋 I see @tnull was un-assigned. |
febyeji
force-pushed
the
add-cbf-chain-source
branch
5 times, most recently
from
6e39d0d to
86e88ce
Compare
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
86e88ce to
3de4777
Compare
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
3de4777 to
9692ff4
Compare
|
🔔 1st Reminder Hey @tnull! This PR has been waiting for your review. |
febyeji
force-pushed
the
add-cbf-chain-source
branch
6 times, most recently
from
388d7d2 to
fdf6710
Compare
Collaborator
|
@febyeji @randomlogin Please let me know when you consider merging of the two PRs completed and this is ready for a first round of review! |
Fixes the problem when the funding tx is not registered for CBF node, for example during splicing. Added macros to skip incompatible with CBF backend tests (which require mempool).
Decrease timeouts to not falsely conclude some tests as passing
Previously we tracked synced height and also had dirty flag for rescan which was made from the genesis block. This was removed in favor of block heights from channel manager and onchain wallet. Safety reorg interval of 7 blocks (look-back) is added. Force tests use CBF config with 1 required peers, as otherwise it would die if one of two required peers went offline (which made impossible testing of scenarios with 2 nodes on which goes offline).
add auto-restart with exponential backoff (#10) * refactor(cbf): extract build_cbf_node helper and add wallet reference * feat(cbf): auto-restart bip157 node with exponential backoff * feat(cbf): add liveness check before returning requester * fix(cbf): clean up scan state on filter scan failure * fix(cbf): add per-block-fetch timeout to wallet sync
The old wait_for_cbf_sync only checked that the onchain sync timestamp advanced, which could false-pass when the timestamp updated but wallet state was still stale. The new version verifies both onchain and lightning wallet syncs completed, and accepts a check closure so callers can assert concrete wallet state (e.g. balance, payment existence). Also simplifies CbfSyncConfig in test setup to use Default (which already sets required_peers: 1 and reasonable timeouts).
febyeji
force-pushed
the
add-cbf-chain-source
branch
from
22cb2c2 to
817e764
Compare
`push` only appends above the tip, so when `recent_history` contained blocks at or below the wallet's current checkpoint height after a reorg, the stale hashes on the wallet checkpoint were never replaced. Switch to `CheckPoint::insert`, which detects conflicting hashes and purges stale blocks, matching bdk-kyoto's `UpdateBuilder::apply_chain_event`. Also clear `latest_tip` on `BlockHeaderChanges::Reorganized` so cached tip state does not point at an abandoned chain. Update the `checkpoint_building_handles_reorg` unit test (added in c1844b3) to exercise the fixed behaviour: a reorg where the new tip is at the same height as the wallet's checkpoint must still result in the reorged hashes winning. Disclosure: drafted with assistance from Claude Code. Co-Authored-By: Claude Opus 4.7 <[email protected]>
| } | ||
|
|
||
| /// Broadcast a package of transactions via the P2P network. | ||
| pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) { |
Contributor
There was a problem hiding this comment.
bip157 version 0.5.0 includes a Package type that may be used here.
@tnull do I understand correctly that Vec<Transaction> is used so the traits are forward compatible for arbitrary package lengths? My crate only supports packages of size 1 and 2, as per the current Bitcoin Core policy.
Collaborator
There was a problem hiding this comment.
Hmm, well, kinda. LDK's trait uses a &[&Transaction] I believe. Generally we prefer to leave any transaction/package validation to Core via. the chain source. Given bip157 is the chain source, it should be fine to use from_vec and log an error if that fails for whatever reason.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add chain syncing via compact block filters (BIP 157/158) using the
bip157crate. Closes #767.Introduces
CbfChainSource: a new chain backend that syncs wallet and Lightning state through compact block filters over the Bitcoin P2P network.Commits
1. Add BIP 157 compact block filter chain source
Core implementation, built up layer by layer:
bip157dependency andCbfSyncConfigconfigurationCbfChainSourcelifecycle (start/stop) with P2P broadcastConfirmtrait integrationCbfSyncConfig,set_chain_source_cbf)TestChainSource::Cbf,wait_for_cbf_sync)2. Add CBF integration tests and documentation
--test-threads=1(exclusive bitcoind P2P port)LDK_TEST_CHAIN_SOURCEenv var for test configurabilityDesign decisions
bip157crate #767 discussion). Uses block-level averages, which can underestimate during high congestion. (Percentile selection partially mitigates this?)sync_wallets()) + optional background sync viaBackgroundSyncConfig--test-threads=1(exclusive bitcoind P2P port access)